Skip to content

Hand parse-only Kafka header values to extraction as views - #13620

Draft
dougqh wants to merge 2 commits into
dougqh/datadog-id-viewsfrom
dougqh/kafka-value-views
Draft

dougqh wants to merge 2 commits into
dougqh/datadog-id-viewsfrom
dougqh/kafka-value-views

Conversation

@dougqh

@dougqh dougqh commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

What Does This Do

The Kafka 3.8+ consumer's extraction hands the Datadog trace id, parent id and sampling priority headers' values as views of the record's bytes (AsciiSequence, #13617) instead of decoding them into Strings, so the Datadog extractor parses them in place (#13619). Extracted contexts are unchanged.

Stacked on #13619. Part 5 of the value-views series (#13616, #13617, #13618, #13619). Includes #13617's commit (Add AsciiSequence), cherry-picked because #13617 is based on master; it drops out on rebase once #13617 lands.

Additional Notes

🤖 Generated with Claude Code

@dougqh dougqh added tag: performance Performance related changes type: refactoring inst: kafka Kafka instrumentation tag: ai generated Largely based on code generated by an AI or LLM labels Nov 21, 2026
@dougqh
dougqh force-pushed the dougqh/kafka-value-views branch from dc3fa0c to ff9248d Compare November 21, 2026 15:03
@dougqh
dougqh force-pushed the dougqh/datadog-id-views branch from 60b0a1f to 34e9eee Compare November 21, 2026 15:11
@dougqh
dougqh force-pushed the dougqh/kafka-value-views branch from ff9248d to c93abef Compare November 21, 2026 15:11
@datadog-prod-us1-6

datadog-prod-us1-6 Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 59.64% (+0.02%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: cf19f37 | Docs | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.95 s 14.67 s [+0.9%; +2.9%] (maybe worse)
startup:insecure-bank:tracing:Agent 13.75 s 13.69 s [-0.7%; +1.5%] (no difference)
startup:petclinic:appsec:Agent 17.24 s 17.17 s [-0.3%; +1.1%] (no difference)
startup:petclinic:iast:Agent 17.16 s 17.13 s [-0.6%; +1.1%] (no difference)
startup:petclinic:profiling:Agent 16.58 s 16.98 s [-3.4%; -1.3%] (significantly better)
startup:petclinic:sca:Agent 17.28 s 17.10 s [+0.3%; +1.7%] (maybe worse)
startup:petclinic:tracing:Agent 16.39 s 15.96 s [-1.7%; +7.1%] (no difference)

Commit: cf19f372 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@pr-commenter

pr-commenter Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch dougqh/datadog-id-views dougqh/kafka-value-views
git_commit_date 1795289484 1795289486
git_commit_sha 11bb1c7 cf19f37
See matching parameters
Baseline Candidate
ci_job_date 1795290858 1795290858
ci_job_id 2286652610 2286652610
ci_pipeline_id 155480147 155480147
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.31 11.0.31
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=15002 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=15002 -Dhttp.nonProxyHosts=localhost *.localhost
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-sa68w5pw 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-sa68w5pw 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu 11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@pr-commenter

pr-commenter Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch dougqh/datadog-id-views dougqh/kafka-value-views
git_commit_date 1795289484 1795289486
git_commit_sha 11bb1c7 cf19f37
See matching parameters
Baseline Candidate
ci_job_date 1795291051 1795291051
ci_job_id 2286652611 2286652611
ci_pipeline_id 155480147 155480147
cpu_model Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
jdkVersion 11.0.31 11.0.31
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=15002 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=15002 -Dhttp.nonProxyHosts=localhost *.localhost
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-pqzxxn9o 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-pqzxxn9o 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu 11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

dougqh and others added 2 commits November 21, 2026 14:31
The byte counterpart of SubSequence: a view of a range of an array whose
bytes are all ASCII, without copying or decoding them, so a value that
arrived as bytes can be parsed in place. ofAscii returns null if a byte
isn't ASCII, and the caller decodes as before. @NoEscape, like
SubSequence: a view of a fetched record's bytes shares the whole fetch
buffer, so a value that must be kept is copied with toString().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the visitor takes transient views (the composite propagator's cache
does) and values are plain UTF-8, the Kafka 3.8 adapter hands a fetched
header's parse-only value (the Datadog trace id, parent id and sampling
priority) as an AsciiSequence over its bytes instead of decoding it, so
the Datadog extractor parses it in place. Non-ASCII values, direct
buffers, Base64 mode and other visitors take the String path as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dougqh
dougqh force-pushed the dougqh/kafka-value-views branch from 08e45a4 to cf19f37 Compare November 21, 2026 19:39
@dougqh
dougqh force-pushed the dougqh/datadog-id-views branch from 06bd242 to 11bb1c7 Compare November 21, 2026 19:39

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

inst: kafka Kafka instrumentation tag: ai generated Largely based on code generated by an AI or LLM tag: performance Performance related changes type: refactoring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant